Skip to main content

View Post [edit]

Poster: Javik Date: Nov 23, 2006 9:59pm
Forum: petabox Subject: FYI, don't need more PCI slots, just SATA-II or SAS

For people building huge arrays of SATA drives, and then using a huge pile of SATA controllers running one drive cable back to a controller port... you don't really need to do that. You do not need a 14-slot backplane to power a whole pile of SATA controllers. It is possible to use a SINGLE port on a SATA-II (3.0 gbps) controller for up to 15 drives, using an obscure piece of hardware you typically will not find standalone, known as a "Port Expander". So, a four port SATA-II controller is potentially capable of managing 60 hard drives. (!) I have no idea why, but I can only find 1-to-5 SATA-II port expanders. I've searched high and low and I cannot find anyone making the 1-to-15 version, so you're sorta screwed here. (Maybe your Google-fu is better than mine.) For example: SATA 5 Port Multiplier HUB eSATA Ports, $100 Note, for this to work you MUST use a SATA-II/SATA 3.0gbps board and it must explicitly say in the description that the BIOS fully supports use of expanders. So, 15 devices per port not good enough for you? Then it is time to move up to SAS, or Serial-Attached SCSI. This bad boy supports 128 drives per port, using something known as an "SAS Edge Expander". This can go further yet, by using an "SAS Fan Out Expander" to split the signal across multiple Edge Expanders. The maximum spec supports up to 128 Edge Expanders, permitting a total of .... 16,384 SATA hard drives on ONE channel. ONE PCI SLOT! Note this is still all an extremely obscure topic that you will only be able to find out if you dig through the design specifications for the SATA-II standard and various engineering magazines. And though the spec is in place, it's not like anyone is really actually taking it to the limit just yet. It is still expensive, for now.... RTSASRFO-12X 12-Port SAS Fan Out Expander, $1,225.00 MODEL: RTSASR-12X, 12-port SAS Edge Expander, $1,195.00 So what does all this stuff look like in a real server application? Well, in many cases the fragile little individual SATA cable plugs are being beefed up for external [ab]use as well as "multilane" merged internal cable assembles that bring many channels together into one socket: -Javik
This post was modified by Javik on 2006-11-24 05:13:53
This post was modified by Javik on 2006-11-24 05:59:18

Reply [edit]

Poster: Javik Date: Nov 25, 2006 6:15pm
Forum: petabox Subject: 24-drives on one SAS channel, from SuperMicro

There is, however, another way to get your hands on SAS expanders that are not so expensive. SuperMicro for example, is making a SATA hot-plug drive enclosure which incorporate SAS expanders right into the enclosure. You just keep plugging the enclosures together in a long chain. SuperMicro CSE-M28E1 SAS SATA HDD TRAY 8X2.5IN 1X EXPANDER, $335 Look at the bottom of the page and you can see they show how you can daisychain three of these, for 24 2.5" SATA drives on a single SAS controller.
This post was modified by Javik on 2006-11-26 02:15:04

Reply [edit]

Poster: Javik Date: Nov 25, 2006 6:14pm
Forum: petabox Subject: Oh look, the first ever 2.5 inch SAS hard drive

Hmm, okay, so this is sort of new. Bleeding-edge new. :) Fujitsu MAV2073RC 74GB 10,000 RPM 8MB Cache Serial Attached SCSI (SAS) Hard Drive, OEM, $639 Features: * Mean-time-between-failure (MTBF) of 1,400,000 power-on hours * The industry's first 2.5-inch small-form-factor serial-attached SCSI (SAS) interface * Native Command Queuing, providing faster execution of operation commands * Uses smaller connectors and cables than U320, a benefit acknowledged by the SCSI Trade Association (STA) -------------- But for high-speed access, smaller is better since the head-arm doesn't have to cover such a large area, and can be lower mass for greater movement speed: Average Seek Time: 4ms Average Write Time: 4.5ms Average Latency: 2.99ms A smaller platter also means they can spin it faster for even higher throughput. Once this technology has had time to mature it may eventually move up to 20,000 RPM platters. -Javik
This post was modified by Javik on 2006-11-26 02:14:07

Reply [edit]

Poster: Javik Date: Nov 29, 2006 12:34pm
Forum: petabox Subject: Re: Make that --144-- 2.5" drives on one SAS channel

Heh, SuperMicro has some really powerful products but their product info pages do not do a good job of explaining the full capabilities of the hardware.

If you take a look at the manual for the E1/E2 chassis you will see that it's not limited to 24 drives, but in fact up to 144 drives:

Page 4 says:
w/potential support of up to 144 SAS devices (*Note)

*Note: The availability of SAS-devices supported depends on the readiness of firmware and hardware support. The firmware support and the hardware support for SAS devices are not validated at the time of publishing of this manual. It's subject to change.

And it seems that the main difference between the E1 and the E2 is that, as with the SC933 in my other post, the E2 has a redundant data channel to be used with SAS hard drives that have dual I/O ports on them.

So it looks to me that if you were to use just cheap SATA-II drives, you only need the E1 SAS backplane, as cheap SATA-II drives don't have the redundant I/O port available on the hotplug connector in the E2 chassis.

On page 8 of the backplane diagram, the E1 just has the four ports on the right side. The E2 has both the four on the right and the four on the left.

When cascading (daisy-chaining) the E1 chassis, two ports go to the SAS controller, and two ports are pass-through to the next E1 chassis.

When cascading (daisy-chaining) the E2 chassis, two ports go to one SAS controller, two ports to a second SAS controller, and four ports are pass-through to the next E1 chassis.

Meanwhile, if you only use one chassis, you can use all four (on the E1) or all eight (on the E2) to connect the single chassis to a single SAS card, though why you might do this is not explained.

I am assuming that when multiple SAS channels are used in parallel, it gives better performance. Perhaps it permits parallel drive accesses at 3.0Gbit, but again it is not explained.

An engineering degree in SAS technology may be required to understand all the hardware options available here. :-)


Reply [edit]

Poster: Gobba Date: Apr 14, 2011 4:48am
Forum: petabox Subject: Re: FYI, don't need more PCI slots, just SATA-II or SAS

This is a really old thread, but i would like to know what to use today to use lots of disks really cheap?